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Abstract. The paper gives the complete characterization of all 
graded nilpotent Lie algebras with infinite-dimensional Tanaka pro- 
longation as extensions of graded nilpotent Lie algebras of lower 
dimension by means of a commutative ideal. We introduce a no- 
tion of weak characteristics of a vector distribution and prove that 
if a bracket-generating distribution of constant type does not have 
non-zero complex weak characteristics, then its symmetry algebra 
is necessarily finite-dimensional. The paper also contains a num- 
ber of illustrative algebraic and geometric examples including the 
proof that any metabelian Lie algebra with a 2-dimensional center 
always has an infinite-dimensional Tanaka prolongation. 



1. Introduction 

This paper is devoted to the study of non-holonomic vector distribu- 
tions with infinite-dimensional symmetry algebras. The simplest exam- 
ples of such distributions are contact distributions on odd-dimensional 
manifolds and, more generally, contact systems on jet spaces J k (M. n , R m ). 
The symmetry algebras of such distributions are given by Lie-Backhand 
theorem and are isomorphic to either the Lie algebra of all vector fields 
on J°(M n ,R m ) = R n+m for m > 2 or to the Lie algebra of all contact 
vector fields on J 1 (M ri ,M) for m = 1. In both cases the symmetry 
algebras are infinite-dimensional. 

We are interested in only so-called bracket- generating distributions, 
i.e., we always assume that repetitive brackets of vector fields lying in 
D generate the whole tangent bundle TM. If this is not the case, then 
D lies in a certain proper completely integrable vector distribution D', 
and the geometry of D can be essentially reduced to the restrictions of 
D to the fibers of D'. 

We shall also say that the distribution D is degenerate, if it possesses 
non-zero Cauchy characterisitics, i.e., vector fields X 6 D such that 
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[X, D] C D. It is also clear that any degenerate distribution, even if 
it is bracket-generating, has an infinite-dimensional symmetry algebra. 
For the proof and other basic properties of vector distributions we refer 
to |2l Chapter 2]. 

While it seems to be very difficult to provide the complete local de- 
scription of all vector distributions with infinite-dimensional symmetry 
algebra, it appears that it is possible to give the complete description 
of the their symbol algebras. 

Namely, let D be a bracket-generating vector distribution on a smooth 
manifold M . Taking repetitive brackets of vector fields lying in D, we 
can define a weak derived flag of D: 

0cDcD 2 C---CD tM = TM, 

D° = 0, D 1 = D, D i+1 = [D, D% i > 2. 

At each point p G M we can defined the associated graded vector 
space 

(1) m(p) = J>_,(p), m_,(p) = D p /D p -\ 

i<0 

Since [D % 1 D^] C D %Jr i for all i,j > 0, m(p) is naturally equipped with 
a structure of a graded Lie algebra. Namely, if x G m_j(p) and y G 
m_j(p) are two homogeneous elements in m(p), and X G D\ Y G D-* 
are two vector fields such that X p + D l ~ l = x and Y p + Dp' 1 = y, 
then the value of [X, Y] + D % ^i~ x depends only on x and y, and, thus, 
defines a graded Lie algebra structure on m(p). It is also clear from the 
definition, that m(p) is a nilpotent Lie algebra generated by m_i(p). 

This Lie algebra is called a symbol of the distribution D at a point 
p G M, and it plays essential role in study of D and any geometric 
structures subordinate to D. For example, if D is a contact struc- 
ture on a smooth manifold of dimension In + 1, i.e., a non-degenerate 
codimension 1 distribution on M, then its symbol is isomorphic to the 
(2n + l)-dimensional Heisenberg Lie algebra at any point p G M. 

The family of graded Lie algebras xn(p) is a basic invariant of any 
bracket-generating distribution, which includes not only the dimensions 
of the weak derived series of D, but also a non-trivial algebraic infor- 
mation. In many cases the structure of these algebras has very impor- 
tant geometric consequences and allows to associate various geometric 
structures with M. One of the most famous examples is E. Cartan 
paper [3] , where he associates a GVgeometry with any non-degenerate 
2-dimensional vector distribution on a 5-dimensional manifold. 

We say that the distribution D has constant symbol m or is of type 
m if its symbols m(p) are isomorphic to m for all points p G M. For 
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example, the contact distribution and all contact systems on jet spaces 
are of constant type. 

We shall use the term graded nilpotent Lie algebra or simply GNLA 
for any negatively graded Lie algebra m = YHt=\ m -« generated by m_i. 
We shall call such Lie algebra non-degenerate, if m_i does not include 
any non-zero central elements. This has a clear geometric meaning. It 
is easy to see that the symbol of a bracket-generating distribution D is 
non-degenerate if and only if D has no non-zero Cauchy characteristics. 
The largest /i such that ^ is called the depth of m. 

It appears that we can derive the exact bound for the dimension of 
the symmetry algebra of D purely in terms of its symbol m. Namely, 
Tanaka [15] has shown in his pioneer works on the geometry of filtered 
manifolds, that there is a well-defined graded Lie algebra g(m) called 
Tanaka (or universal) prolongation of m. It is characterized by the 
following conditions: 

(1) 0i(tn) = rrij for all i < 0; 

(2) if [X, m] = for certain X e 0;(m), then X = 0; 

(3) £j(m) is the largest graded Lie algebra satisfying the above two 
conditions. 

The Lie algebra fl(m) has also a clear geometric meaning. Namely, let 
M be a connected simply connected Lie group with the Lie algebra 
m. For example, we can identify M with m and define the Lie group 
multiplication in M by means of Campbell-Haussdorf series. Define the 
distribution D on M by assuming that it is left invariant and that D e = 
m_i. Then fj(m) can be naturally identified with the graded Lie algebra 
associated with the filtered Lie algebra of all germs of infinitesimal 
symmetries of D at the identity. In particular, if $j(m) is finite or 
infinite-dimensional, so is the symmetry algebra of the corresponding 
filtration. Such distributions D are called standard distributions of 
type m. 

One of the main result of Tanaka paper [T5] can be reformulated as 
follows. 

Tanaka theorem Q15J). If g(m) is finite- dimensional, then with each 
distribution D C TM of type m we can associate a canonical coframe 
on a certain bundle P over M of dimension dimg(m). In particular, 
the symmetry algebra of the distribution D is finite-dimensional, and 
its dimension is bounded by dim0(m). 

Thus, if D is a holonomic distribution with infinite-dimensional sym- 
metry algebra, then the Tanaka prolongation of its symbol fl(m) should 
also be infinite-dimensional. The properties of Tanaka prolongation are 
also studied in [TBI H3 EH UM CI] • 
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The main result of this paper is Theorem (U which gives the the com- 
plete characterization of all GNLA with infinite-dimensional Tanaka 
prolongation as extensions of graded nilpotent Lie algebras of lower di- 
mension by means of a commutative ideal. Along with this description 
we also introduce a notion of weak characteristics of a vector distri- 
bution and prove that if a bracket-generating distribution of constant 
type does not have non-zero complex weak characteristics, then its 
symmetry algebra is necessarily finite-dimensional. 

This article is closely related to a series of papers devoted to the ge- 
ometry of 2 and 3-dimensional distributions [UIUE]. These papers show 
that under very mild non-degeneracy conditions the symmetry algebra 
of a non-holonomic vector distribution becomes finite-dimensional. As 
we show in Theorem [2J the set of all GNLA, whose Tanaka prolonga- 
tion is infinite-dimensional, forms a closed subvariety in the variety of 
all GNLA m with fixed dimensions of subspaces m_j, i > 0. 

Finally, in the last section of the paper we present a number of 
illustrative algebraic and geometric examples including the proof that 
any metabelian Lie algebra with a 2-dimensional center always has an 
infinite-dimensional Tanaka prolongation. 

We would like to express our gratitude to Ian Anderson, Tohru Mo- 
rimoto, Ben Warhurst, Igor Zelenko for stimulating discussions on the 
topic of this paper. 

2. Tanaka and Spencer criteria 

Below we assume that all Lie algebras we consider are defined over 
an arbitrary field k of characteristic 0. We try to be as generic as 
possible in our algebraic considerations and we explicitly state, when 
we require the base field k to be algebraically closed. 

Let m be an arbitrary GNLA and let g(m) be its Tanaka prolon- 
gation. We say that m is of finite (infinite) type, if g(m) is finite- 
dimensional (resp., infinite-dimensional). Note that this notion is sta- 
ble with respect to the base field extensions, since each subspace 0j(m), 
% > of the Tanaka prolongation can be computed by solving a system 
of linear equations. Namely, assuming that all subspaces 0j(m), i < k 
are already computed, the space fl fc (m) can be defined as follows: 

flfe(m) = {<p: m ->• y^&(m) | 0(m_i) c 0k-t(m), 

i<k 

0(^2/]) = [<P(z),v] + [x,<P(y)],Vx,y e m}. 

Tanaka criterium reduces the question whether Tanaka prolongation 
g(m) of m is finite-dimensional or not to the same question for the 
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standard prolongation (as described, for example, in [Hj) of a certain 
linear Lie algebra. Namely, let Dero(m) be the Lie algebra of all degree- 
preserving derivations of m. Define the subalgebra h C Der (m) as 
follows: 

f)o = {d G Der (m) | d(x) = for all x G m_j, i > 2}. 

We can naturally identify f) with a subspace in End(m__i). Then 
Tanaka criteruim can be formulated as follows: 

Tanaka criterium ([15]). Tanaka prolongation g(m) of m is finite- 
dimensional if and only if so is the standard prolongation of F)o C 
End(m_i). 

In its turn, Spencer criterium provides a computationally efficient 
method of detecting whether the standard prolongation of a linear sub- 
space A C End(V) is finite-dimensional or not. 

Spencer criterium ([HUE!)- The standard prolongation of a subspace 
A C End(V) is finite- dimensional if and only if A k C End(V k ) does 
not contain endomorphisms of rank 1 . 

Here by k we mean the algebraic closure of our base field k, and 
by A k the subspace in End(V fc ) obtained from A by field extension. 
A detailed and self-contained proof of Spencer criterium can be found 
in [12]. 

We emphasize that this criterium if computationally efficient, as the 
set of all rank 1 endomorphisms in A is described by a finite set of 
quadratic polynomials, and, for example, Grobner basis technique pro- 
vides an algorithm to determine whether this set of polynomials has a 
non-zero common root. 

In the next section we use both these criteria to prove that all fun- 
damental Lie algebras m of infinite type over an algebraically closed 
field possess a very special algebraic structure. 

3. Graded nilpotent Lie algebras of infinite type 

Let g be an arbitrary finite-dimensional Lie algebra, and let V be 
an arbitrary g-module. We recall that a Lie algebra g is called an 
extension of g by means of V, if g can be included into the following 
exact sequence: 

O^V ^ g ^ 0^0. 

In other words, V is embedded into g as a commutative ideal, the quo- 
tient g/V is identified with g and the natural action of g = g/V on V 
coincides with the predefined g-module structure on V. Two extensions 
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gi and g 2 are called equivalent, if there exists an isomorphism gi — > g 2 
identical on V and g = Qi/V = Q 2 /V . 

It is well-known that equivalence classes of such extensions are de- 
scribed by the cohomology space H 2 (q, V). Namely, if [a] is an element 
of H 2 (q,V), where a G Z 2 (q,V), then g can be identified as a vector 
space with g x V with the Lie bracket given by: 

(2) [(xiiVt), {x 2 ,v 2 )) = ([xi,x 2 ),x 1 .v 2 - x 2 .vi + a(xi,x 2 )), 

x x ,x 2 e g, vx,v 2 e V. 

Assume now that both the Lie algebra g and the g-module V are 
graded, that is g = J^Qi, ^ = Yl Vj an d Qi-Vj C V i+ j. Then the 
cohomology space H 2 (g, V) is naturally turned into the graded vector 
space as well: 

h 2 ( q ,v) = J2hKq,v). 

It is easy to see that the Lie algebra g defined by (j2J) is also graded if 
and only if [a] G Hq(q,V). Thus, we see that the graded extensions 
g of the graded Lie algebra g by means of the graded g-module V are 
in one to one correspondence with the elements of the vector space 
H 2 (9,V). 

We can introduce an additional group action on H 2 (g, V) as follows. 
Define Aut (g, V) as a subgroup in Aut (g) x GL (V) of the form: 

Aut (g, V) = {(f,g) G Aut (g) x GL (V) | 

g (x.v) = f(x).g(v), Vx G g,v G V}. 

This group naturally acts on Hq(q, V): 

(f,g).[a] = [goaof- 1 ]. 

It is easy to see that elements of Hq(q, V) lying in the same orbit of 
this action correspond to the isomorphic Lie algebras. 

In this paper we shall consider special extensions of GNLA. Namely, 
let n be an arbitrary (possibly degenerate) GNLA. Fix any subspace 
W C n_i and consider ti-i/W as a graded commutative Lie alge- 
bra concentrated in degree — 1. Take any graded n_i/W-module V = 
~52i = i V-i generated by V-\ (as a module). Assuming that W.V = 
and n_i.V = for alH > 2 we can consider V as an n-module. We call 
any extension of the GNLA n by means of such n-module V a special 
GNLA extension. Let [a] be the corresponding element of H^(n, V). It 
is easy to see that such extension is non-degenerate if and only if the 
following two conditions hold: 

(1) the condition n.v = implies v = for v G V-x, 

(2) the cocycle a is non-degenerate on Z(n) PI n_i. 
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We note that unlike the extensions of GNLA considered in the pa- 
pers pUHl], the special extensions defined above are not central exten- 
sions of Lie algebras, as the ideal V above does not in general belong 
to the center, and the action of n on V is non-trivial. 
The main result of the paper is: 

Theorem 1. Let m be a non- degenerate GNLA oven an algebraically 
closed field. Then the following conditions are equivalent: 

(1) there exists such d £ Der (m) that <i(m_j) = for i > 2 and 
rank d = 1 ; 

(2) there exists such y £ m_i that rank ad y = 1; in particular, in 
this case [y, m_j] = for all i > 2; 

(3) m can be represented as a special extension of a certain (possibly 
degenerate) GNLA n with dimm_i = dimn_i + l and dim V_i = 
1 for all i = —1, . . . , — fi and fi > 2; 

(4) m is of infinite type. 

Proof. (1) =>- (2). Let f) be a subalgebra in Der (m) defined by con- 
dition d(m.-i) = for any d £ f). Let d £ f) and rankrf = 1. Define 
W C m as W = kerd. Choose non-zero x,y £ m_i such that d(x) = y. 
Then we have m_i = (x) © W, and W D [m, m] . Next, 

\y,W] = [d(x),W] = -[x,d(W)] = 0. 

Since m is non-degenerate, this implies that rank &dy = 1 and [x,y] ^ 
0. We also have 

[x,d(y)\ = -[d(x),y] = -[y,y] = 0. 
On the other hand, 

[d(y),W] = -[y,d(W)} = 0. 

Hence, [d(y),m] = 0, and from non-degeneracy of m it follows that 
d(y) = 0. In particular, d is nilpotent. 

(2) =>■ (1). Let y £ m_i such that rank ad y = 1. Let W = ker&dy. 
Choose such x £ m_i that [x,y] ^ 0. Such x always exists by non- 
degeneracy of m. It is clear that m = (x) © W and y £ W. It is also 
clear that W D [m, m]. Define d £ f) as d(x) = y and d(W) = 0. It is 
easy to see that d is indeed a differentiation of m. 

(2) ^> (3). Again, let x, y be elements in m_i such that rank ad y = 1, 
[x,y] ^ 0, tn = W D [m, m] and [y, W] = 0. Set yi = y and 

define = [x, y^] for all i > 2. Note that y 2 7^ 0. Let /x be the 
smallest integer such that y M ^ and y^+i = 0. By induction we get: 

[y i+1 , W] = [[x, yi },W] = [x, [W, Vi \] + [y h [x, W}] = 0. 
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Let V = (y h . . . y M . It is clear that V C W, [V, W] = 0] and [x, V) C V. 
Hence, V is a commutative ideal in m. Put n = m/V. It is easy to see 
that V is generated by V-\ as an n-module and the action of [n, n] on 
V is trivial. Finally, it is evident that n is also generated by n_i as a 
graded Lie algebra. Thus, m is represented as a special extension: 

(3) -> V ->• m -> n -> 0. 

(3) =>- (2). Let m be a special extension ([3]) of n with dimV.j = 1, 
i = 1, • • • Take ?/ as a non-zero element in VLi. Since [n, n] acts 
trivially on V, we see that [y,m] C V-2- Since is generated by V-\ 
as an n-module, we see that ady 7^ and rank ad y = 1. 

(4) <^ (1). This is exactly the combined Tanaka and Spencer criteria. 

□ 

Remark 1. Only the last implication uses the fact that the base field 
is algebraically closed. In case of arbitrary field of characteristic the 
items (1), (2) and (3) are still equivalent and imply that m has infinite 
type, a simple counterexample for the implication (4) =>- (1) over M is 
given by the 3-dimensional complex Heisenberg Lie algebra (with an 
obvious grading of depth 2) viewed as a 6-dimensional real Lie algebra. 

Remark 2. Condition (2) of the Theorem appears already in [11J as a 
sufficient condition for m to be of infinite type. 

Item (3) of Theorem [1] gives an inductive algorithm for constructing 
all GNLA of infinite type. Namely, to construct all GNLA m of infinite 
type and dimm_i = n + 1 one needs to: 

• take an arbitrary GNLA n (of finite or infinite type) such that 
dimn_i = n; 

• take an arbitrary subspace W C n_i of codimension 1; 

• fix any integer s > 2 and construct a graded n_i/H^-module 
V = V-h which is uniquely defined by the conditions 
dimK_j = 1, % = 1, . . . , s and = (n_i/W).VL.j; 

• compute cohomology space Hq(vi, V), where V is treated as an 
n-module with the trivial action of W © [n, n]; 

• take any uj G Hq(xx, V) and define m as an extension of n by 
means of V corresponding to u. 

In coordinates this procedure can be reformulated as follows. Fix any 
convector a G n*^ and an element X G n_i such that a(X) = 1. Ex- 
tend X to the basis {X, Z x , . . . , Z r } of n consisting of homogeneous ele- 
ments. Define m as a Lie algebra with a basis {X, Yi, . . . , Y s , Z\, . . . , Z r }, 



GRADED NILPOTENT LIE ALGEBRAS OF INFINITE TYPE 



9 



s > 2, where 

[X,Y i \=Y i+ll i = l,...,s-l; 
[Zj,Yi\ = 0, i = l,...,s, j = l,...,r; 
[Ki,^-] = 0, i,j = l,...,s; 

s 

[Zj, Zj] m = [Zj, Zj] n + 

fc=l 

where the constants a\ and define a cocycle A 2 n — > V of degree 
(i.e., m is a graded Lie algebra) and are viewed modulo the changes 
of the basis X X + J2i=i c i^s, Z% Zi + J2j=i d\Yj. See the next 
section for the concrete examples. 

Theorem [I] provides also an easy way to prove that the set of all 
graded Lie algebras of infinite type forms forms a closed algebraic sub- 
variety in the variety of all graded nilpotent Lie algebras. Namely, fix 
a sequence of integers k%, . . . , k^ and denote by M(ki, . . . , k^) a vari- 
ety of all graded nilpotent Lie algebras m such that dimm_j = ki for 
i = 1, . . . , /i and m_j = for i > \i. It is easy to see that it is a closed 
algebraic variety in the vector space of all skew-symmetric graded al- 
gebras (without any conditions on multiplication). 

The set of all fundamental graded Lie algebras in Mikj) is distin- 
guished by conditions [m_i , m_j] = m_j_i for all % — 1, . . . , [A — 1. It 
is easy to see that these conditions define an open subset (in Zarisski 
topology) in M(fcj), which we shall denote by MF(ki). 

Define MF°°(ki) as the set of all fundamental Lie algebras of infinite 
type. 

Theorem 2. MF°°(ki) is a closed subvariety in MF(ki). 

Proof. The existence of an element y G m_i such that rank ad y = 1 
(and, hence, [y, m_;] = for i > 2 can be reformulated as an existence 
of a non-zero solution of a certain system of quadratic equations (2 by 2 
minors of the matrix of ady), whose coefficients are structure constants 
of the Lie algebra m. It is well-known [8, Theorem 3.12] that that the 
conditions when such non-zero solution exists are given in terms of the 
algebraic equations on the coefficients of this system. □ 

Theorem [1] also has a number of immediate geometric applications. 
Namely, let Dbea bracket generating distribution with a constat sym- 
bol m. As usual, we assume that D has no Cauchy characteristics. This 
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implies that m is non-degenerate, i.e., does not have non-zero central 
elements lying in m_i. 

We say that a vector field Y G D is a weak characteristic of D, if 
the following conditions hold: 

(1) Y G char(£) i ) for all i > 2, or, in other words, [Y, (£>*)] C (£>*) 
for all z > 2; 

(2) the subbundle D' C D spanned by all X G D such that [X, Y] C 
D has codimension 1 in D. 

Similarly, we can define complex weak characteristics of a vector distri- 
bution D as sections of the comlpexified subbundle D c of the comlpex- 
ified tangent bundle T C M satisfying the conditions (1) and (2) above, 
where we replace powers D l by their complexified versions. 

Theorem 3. Let D be a bracket generating distribution with a con- 
stant symbol m. Assume that D has no Cauchy characteristics and 
that dimsym(D) = oo. Then D contains non-trivial complex week 
characteristics. 

Proof. This immediately follows from item (2) of Theorem [1] and the 
Tanaka theorem [15] that bounds the dimension of sym(D) by the 
dimension of the Tanaka prolongation of its symbol m. □ 

Corollary 1. If D n (flj>2 char(Z) 1 )) = 0, then the symmetry algebra 
of D is finite- dimensional. 

Proof. Assume that sym(D) is infinite-dimensional. Then it possesses 
complex week characteristic Y. Clearly both Y and cY, c G C, are 
also complex week characteristics. Therefore, the complex subbundle 
spanned by Y and Y is stable with respect to the complex conjugation. 
Hence, it also contains a real one-dimensional subbundle spanned by a 
certain non-zero vector field X. While X itself might no longer be a 
week characteristic, it still lies in D fl (n»>2 char(D i )). □ 

Remark 3. We formulate Theorem [2] and its corollary only for the 
vector distribution with constant symbol, since the Tanaka theorem 
on the bound of dimsym(D) has been proved up to now only in this 
case. The Tanaka proof does not immediately generalize to the case of 
non-constant symbol, and this still remains an open problem according 
to the authors knowledge. 

4. Examples 

4.1. Symbols of 2-dimensional distributions of infinite type. 

As a first application of Theorem [1] we describe the symbols of 2- 
dimensional bracket generating distributions of infinite type. 
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Using Theorem (TJ it is easy to prove that in each dimension n > 2 
there exists a unique (up to isomorphism) GNLA m with dimm_i = 2. 
It turns out to coincide with the symbol of the Goursat distribution, 
which can coincides also with the canonical contact system on the jet 
space J"- 2 (M,M). 

Indeed, by Theorem [1] any such GNLA must be a special extension 
of a certain GNLA n with dimn_i = 1. But then n = R is a one- 
dimensional commutative Lie algebra, and all its special extensions are 
trivial. In each dimension there is exactly one such extension. It can 
be described as a Lie algebra m with a basis (X, Z%, . . . , Z n -i) and 
the only non-zero Lie brackets [X, Zi\ = Zi + x, i = 1, . . . , n — 2. Here 
deg X = —1 and deg Zi = —i, i — 1, . . . , n — 1. It is easy to see that the 
contact system on J n_2 (M, R) is equivalent to the standard distribution 
of type m. 

4.2. Symbols of 3-dimensional distributions of infinite type. In 

case of dimm_i = 3 we already get a lot of various examples, including 
the examples of non-trivial special extension of GNLA. As above, any 
GNLA m of infinite type with dimm_i = 3 is a special extension of a 
certain GNLA n with dimn_i = 2. Note that n itself does not need to 
be of infinite type. As the classification of all such GNLA unknown at 
the moment, it makes also impossible to classify all infinite type GNLA 
with dimm__i = 3. 

As an example, let us describe all special extensions in case, when 
n is a three-dimensional Heisenberg algebra. Then all trivial special 
extensions of n can be described as: 

m = (X, Zi, Z 2 , Yi, . . . , Y k ), 

where degX = —1, deg Yj = —i, i = 1, . . . , k, deg Zj = —j, j = 1, 2. 
All non-zero Lie brackets are [X, Zi] = Z 2 and [X, Yj\ = Yi+i, i = 
1, . . . , k — 1. The Lie algebra m is a semidirect product of its subal- 
gebra n = (X,Zi,Z 2 ) and the commutative ideal V = (Y±, . . . , Yk). 
Geometrically the standard distribution of type m can be described as 
a canonical contact system on the mixed jet bundle J 1,fe_1 (]R, M. 2 ). 

Any non-trivial special extension of n can be obtained by adding 
non-trivial Lie brackets of the form: 

[X,Z 1 } = Z 2 + aY 2 ; 
[X,Z 2 ] = bY 3 ; 
[Z 1 ,Z 2 ] = cY 3 . 

By changing Z\ to Z\ — dY\ and Z 2 to Z 2 — bY 2 we can always obtain 
a = b = 0. If a 0, then the Jacobi identity is satisfied only if k = 3. 
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In this case we can always scale a to 1. Thus, we see that up to the 
isomorphism there exists exactly one non-trivial special extension of 
the three-dimensional Heisenberg Lie algebra. It is given by: 

m=(X,Z 1 ,Z 2 ,Y 1 ,Y 2 ,Y 3 ), 

where non-zero Lie brackets are given by: 

[X,Y 1 ]=Y 2 , [X,Y 2 ] =Y 3] 

[X,Z 1 ]=Z 2 ; 

[Z U Z 2 ]=Y 3 . 

Direct computation shows that an arbitrary symmetry of the standard 
distribution of type m depends on 4 functions of 1 variable. 

4.3. Metabelian Lie algebras. Let m = m_i © m_ 2 be a GNLA of 
depth 2. The structure of this Lie algebra is completely determined by 
the skew-symmetric map: S : A 2 m_i — >■ m_ 2 defined as a restriction of 
the Lie bracket to m_i. Below we assume that m is non-degenerate. 

Consider the dual map S* : m* 2 — > A 2 ^:*^. Since m is generated 
by m_i, the map S* is injective. Thus, up to the isomorphism the 
structure of m is determined by the subspace ImS* C A 2 m*_ 1 . This 
subspace is considered up to the natural action of the group GL(m_i) 
on A 2 m*_ 1 . 

To simplify the notation, let V = m_i, n = dimV, and let P = 
ImS* C A 2 V*, m = dim P. If we fix a basis in V, then A 2 V* can 
be identified with a space of all skew-symmetric n by n matrices, P is 
spanned by m matrices B\, ... , B m and the action of GL(V) on A 2 V™ 
is given by: 

X.B = X t BX 1 X G GL(V), B G A 2 V* . 

Let y be an arbitrary element in m_i = V. It is easy to see that 
the rank of ad y is the maximal number of linearly independent skew- 
symmetric matrices B in P, such that i y B ^ 0. In more detail, let P y 
be a subspace in P defined as: 

P y = {B E P \ i y B = 0}. 

Then rank of ady is equal to to codimension of P y in P. 

Theorem 4. Let m = m_i © m_ 2 be a 2-step complex graded nilpotent 
Lie algebra with dimm_ 2 = 2. Then Tanaka prolongation of m is 
infinite- dimensional. 

Proof. Since the dimension of the Tanaka prolongation is preserved 
under extension of the base field, we can assume that our base field is 
algebraically closed. Then according to Theorem [T] m is of infinite type 
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if and only if there exists a non-zero element y G m_i = V such that P y 
is one- dimensional. Let Bi, B 2 be the basis of P. Then the condition 
dim P y = 1 is equivalent to the existence of such Ai, A2 that: 

(Aifli + \ 2 B 2 )y = 0. 

It is clear that this is equivalent to det(Aii?i + A2-B2) — 0, which always 
has non-trivial solutions in case of algebraically closed field. □ 

Note that pencils of skew-symmetric matrices over an algebraically 
closed field can be effectively classified using Kronecker results on pen- 
cils of matrices. This has been done in the work of M. Gauger [6]. 
Namely, any pair (A, B) of skew-symmetric matrices can be written as 
one matrix P = fiA + XB, whose entries are linear forms in A and /i. 
Such matrices are classified by the following data: 

• minimal indices < mi < m% < m p , p > (in particular, the 
set of minimal indices can be empty); 

• elementary divisors (/! + aiA) ei , . . . , (fi + a q X) £r , (A)-^ 1 , . . . , (A)-^ 
(each of these divisors appears twice). 

The canonical form of the pencil P for this data is: 



(M mi 



(4) P 



\ 



Mr 



where 



SJa) 



T 

J n 



^ M m ), (2m+l)x(2m + l),^ = (0); 

J?. (n\\ 

() (2n)x(2n); 
(2n) x (2n); 



E n {a) 
-EJaY 



F n 



-Ft 
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and 



/ 



M„ 



A jj, 

■ n 



A 

A fj, 

( 



, (m + 1) x m; 



E n (a) 



F n = 



ji + a\ 
\fj, + aX A 

/ A \ 



A 

\A 11 



H + a\\ 
A 



, n x n; 



, n x n; 



In addition, the elementary divisors are considered up to non-degenerate 
linear transformations of (A,/i). 

Note also, that the pencil P above corresponds to a non-degenerate 
2-step nilpotent Lie algebra if and only if there are no minimal indices 
among the set of minimal indices mi, . . . ,m p . As we assume that 
these indices are ordered, this is equivalent to the condition mi > 0. 

Define the following subgroups in GL(V) and the corresponding sub- 
algebras in 0l(V): 

Aut(P) = {X e GL(V) | X'BX C P for any B G P}, 
H = {X E GL(V) | X l BX = B for any B E P}, 
Der(P) = {X E gl(V) | X l B + BX C P for any B G P}, 
f)o = {X G gl(V) | X l B + BX = for any £ G P}. 

It is clear that: 
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• Der(P) and f)o are subalgebras in gl(V) corresponding to the 
subgroups Aut(P) and H respectively; 

• H is a subgroup in Aut(P), f)o is a subalgebra in Der(P); 

• Aut(P) is naturally identified with the the group Auto(m) of 
all grading-preserving automorphisms of the Lie algebra m and 
Der(P) is identified with the Lie algebra Der (m) of all grading- 
preserving derivations of m; 

• f)o can be identified with all elements in Der (m) that act triv- 
ially on m_ 2 - 

Let us show explicitly that (jo always contains a rank 1 element. 
This will give another proof that the standard prolongation of f)o is 
infinite-dimensional. 

Let Q be any of blocks that appear on the diagonal in the canonical 
form 01]) of the pencil P. That is Q is one of the following matrices: 
M m for m > 0, £ e (a) for e > 1, or JFf, f > 1. We shall call Q 
an elementary subpencil of the pencil P and denote by f)o(Q) the Lie 
algebra 



It is easy to see that f)o(Q) can be embedded as a subalgebra into f) - 
So, to prove the theorem it is sufficient to show that f)o(Q) contains 
rank 1 element for each elementary subpencil. Moreover, since elemen- 
tary divisors are considered up to linear transformations of (A,/i), we 
can restrict ourselves only to elementary pencils Ai m , m > 1 and J>, 
r > 1. 

In both these cases the subalgebra f)o(<3) is easily computed. Namely, 
denote by Sk,i(C) the set of k x / matrices (a^) such that a>ij = aj+ij+i 
for any 1 < i < k, 1 < j < I. Denote also by T^(C) the set of all 
k x k upper triangular matrices that also lie in Sfc ( &(C). Note that 
both Sk t i(C) and T^(C) contain elements of rank 1 (for example, in the 
upper right corner). 

We have: 



As we see, in both cases fjo(Q) contains a nilpotent element of rank 1. 

4.4. Remarks. 1. It follows form the proof of Theorem H] that f)o 
contains the block diagonal subalgebra, where each block consists of 
all elements in t)o(Q) f° r the primitive subpencils Q. However, f) can 



fj (Q) = {X e gl(k, C) | X l B + BX = for all B e Q}. 
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be larger than this direct sum. For example, this happens in the case 
when there no minimal indices and the set of (A) and (/x) where each of 
these divisors appears with multiplicity 2p and 2q respectively. In this 
case m is isomorphic to the direct sum of two Heisenberg lie algebras 
of dimension 2p + 1 and 2q + 1. The subalgebra f)o is isomorphic to 
sp(2p, C) ©sp(2g, C), while t) (Q) for each primitive Q is just sll(2, C). 

2. It is well-known that there exist 2-step nilpotent Lie algebras with 
3-dimensional center, whose Tanaka prolongation is finite-dimensional. 
The simplest example is a free 2-step nilpotent Lie algebra with 3- 
dimensional set of generators and the 3-dimensional center: 

m_ 1 = (X 1 ,X 2 ,X 3 ), 

m_2 = (X 12 ,X 13 ,X 23 ), 

where [X^X^] = for 1 < i < j < 3. Its Tanaka prolongation is 21- 
dimensional and is isomorphic to so (7, C). In this case the subalgebra 
f)o is trivial. 

We can also generalize this example for arbitrary number of genera- 
tors. Namely, let: 

m_i = (Xi,X 2 , . . . , X k ), 
m- 2 = {Y 1 ,Y 2 ,Y 3 ), 
where all non-zero Lie brackets have the form: 

[X 1 , Xk] = [X 2 , Xk-\] — ■ ■ ■ — Y±; 
[X 1 ,X k -i] = [X 2 ,X k -2] — • •• — Y 2 ; 
[X 2 , Xk] = [X 3 , Xk-i] = ■ ■ ■ = Y 3 . 

Direct computation shows that Tanaka prolongation fl(tn) of m has the 
form: 

• for k = 3, fl(m) =50(7, C); 

• for k = 4, g(m) = sp(6, C); 

• for k = 5, g(m) = m © q , where q = gl(2, C) and o - m odule 
0_i is irreducible; 

• for k > 6 and even, g(m) = m © g , where g = gl(2,C) x C 
and go-module g_i splits into the sum of k/2 gl(2, C)-modules 
(Xi,X k /2+i), i = 1, •• • , k/2 - 1; 

• for k > 7 and odd, g(m) = m © g , where g = C 2 and the 
action of go on g_i diagonalizes in the basis {X±, X 2 , . . . , X n }. 

3. The above example of 2-step nilpotent Lie algebra with 3-dimensi- 
onal center shows that the subalgebra f)o is trivial for generic 2-step 
nilpotent Lie algebras with dimm_ 2 > 3. In particular, a generic 
GNLA of depth 2 with at least 3-dimensional center is of finite type. 
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However, it is still an open problem to classify all 2-step nilpotent Lie 
algebras with infinite-dimensional Tanaka prolongation. 

4. Even though the theorem above is formulated, and proved over C, 
it is in fact true over any field of characteristic 0, and, in particular, over 
R. Indeed, the results of Kronecker and, thus, Gauger hold over any 
algebraically closed filed of characteristic 0. And it is easy to see that 
the dimension of Tanaka prolongation does not change if we extend the 
field of scalars. 

5. Let D be a vector distribution of codimension 2 on a smooth 
manifold M, such that D 2 = TM. Then Theorem H] implies that the 
symbol m(x) of D is of infinite type for each x £ M. However, this 
does not imply that such distributions always have infinite-dimensional 
symmetry algebras. For example, such distributions with the symbol 
corresponding to the pencil M. m are treated in [9]. It is proved that if 
a subbundle D' generated by all weak characteristics of D is bracket 
generating, then the symmetry algebra of D is finite-dimensional. Note 
that in case of the standard distribution D of type m, where m corre- 
sponds to the pencil Ai m , the subdistribution D' is involutive. 
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